{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 最小二乘法小结\n",
    "\n",
    "## 一、最小二乘法简介\n",
    "\n",
    "最小二乘法是用来做函数拟合或者求函数极值的方法。\n",
    "\n",
    "1. 在参数$\\theta_0 \\theta_1$ 取何值时，损失函数 $J(\\theta_0, \\theta_1)$ 最小？\n",
    "2. 代数法解法、矩阵解法\n",
    "3. 局限性和应用场景  \n",
    "   1. 需要求逆矩阵，有可能逆矩阵不存在\n",
    "   2. 样本特征n非常大是，求n*n的逆矩阵，耗时\n",
    "   3. 拟合函数不是线性的，无法使用，需要使用技巧转化为线性\n",
    "   4. 样本量m很小时（m<n），拟合方程欠定，常用的优化方法都无法优化。当m=n，用方程组求解。当m>n时，方程超定，即常用的最小二乘的应用场景\n",
    "\n",
    "## 二、矩阵解法\n",
    "\n",
    "### 2.1 背景\n",
    "\n",
    "1. 函数：$h_\\theta(x_1, x_2, ...x_n) = \\theta_0 + \\theta_{1}x_1 + ... + \\theta_{n-1}x_{n-1}$,     矩阵表达式：\n",
    "\n",
    "$$\n",
    "h_\\mathbf{\\theta}(\\mathbf{x}) = \\mathbf{X\\theta}\n",
    "$$\n",
    "\n",
    "2. $h_\\mathbf{\\theta}(\\mathbf{X})$：mx1维向量      $\\theta$为nx1维向量      $X$为mxn维的矩阵,   m代表样本个数，n代表样本特征数    $Y$是样本的输出向量mx1维\n",
    "\n",
    "3. 损失函数：\n",
    "\n",
    "$$\n",
    "J(\\mathbf\\theta) = \\frac{1}{2}(\\mathbf{X\\theta} - \\mathbf{Y})^T(\\mathbf{X\\theta} - \\mathbf{Y})\n",
    "$$\n",
    "\n",
    "### 2.2 算法原理\n",
    "\n",
    "1. 对损失函数求导，并令其为0 ：\n",
    "\n",
    "$$\n",
    "\\frac{\\partial}{\\partial\\mathbf\\theta}J(\\mathbf\\theta) = \\mathbf{X}^T(\\mathbf{X\\theta} - \\mathbf{Y}) = 0\n",
    "$$\n",
    "\n",
    "2. 求解该式：\n",
    "\n",
    "$$\n",
    "\\mathbf{X^{T}X\\theta} = \\mathbf{X^{T}Y}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\mathbf{\\theta} = (\\mathbf{X^{T}X})^{-1}\\mathbf{X^{T}Y}\n",
    "$$\n",
    "\n",
    "\n",
    "\n",
    "### 三、代码演示\n",
    "\n",
    "sklearn 的LinearRegression用的就是最小二乘法求解\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1941daa5408>]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3AAAAHSCAYAAACtoSkbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3iW1cHH8e8hgMaJlDjAXW3UqgUb91ut2te4xVFF3FVx1LqjglDfMtUAiqiIiiiKigPR1tY467YKooIj4sCRKKAYRIgYwnn/uINFDELIuJ8n+X6uK1fynGf9SC6RX865zwkxRiRJkiRJma9V2gEkSZIkSSvGAidJkiRJWcICJ0mSJElZwgInSZIkSVnCAidJkiRJWcICJ0mSJElZonXaAWrToUOHuOmmm6YdQ5IkSZJSMWnSpC9jjHlLj2dkgdt0002ZOHFi2jEkSZIkKRUhhI9rG3cJpSRJkiRlCQucJEmSJGUJC5wkSZIkZQkLnCRJkiRlCQucJEmSJGUJC5wkSZIkZQkLnCRJkiRlCQucJEmSJGUJC5wkSZIkZQkLnCRJkiRlCQucJEmSJGUJC5wkSZIkZQkLnCRJkiRlCQucJEmSJGUJC5wkSZIkZYnWaQeQJEmSpKY0YXIZxSWllFdU0rFdLkWF+XTt0intWCvEAidJkiSpxZgwuYye46dQWVUNQFlFJT3HTwHIihLnEkpJkiRJLUZxSekP5W2xyqpqiktKU0pUNxY4SZIkSS1GeUVlncYzjQVOkiRJUovRsV1uncYzjQVOkiRJUotRVJhPbpucH43ltsmhqDA/pUR14yYmkiRJklqMxRuVuAulJEmSJGWBrl06ZU1hW5pLKCVJkiQpS1jgJEmSJClLWOAkSZIkKUtY4CRJkiQpS1jgJEmSJClLWOAkSZIkKUtY4CRJkiQpS1jgJEmSJClLWOAkSZIkKUtY4CRJkiQpS1jgJEmSJClLLLfAhRA2CiE8HUJ4J4TwVgjh3Jrx9iGEx0MI02o+r7OM559Y85hpIYQTG/oPIEmSJEktxYrMwC0ELowxbg3sAvw5hLANcCnwZIxxS+DJmts/EkJoD1wO7AzsBFy+rKInSZIkSfp5yy1wMcbPY4yv1Xw9F3gH6AQcCtxe87Dbga61PL0QeDzGODvG+DXwOLBfQwSXJEmSpJamTtfAhRA2BboA/wHWizF+DknJA9at5SmdgE+XuP1ZzZgkSZIkqY5WuMCFENYAHgDOizF+s6JPq2UsLuP1e4QQJoYQJs6aNWtFY0mSJEnSyvn227QT1NkKFbgQQhuS8jY2xji+ZnhGCGGDmvs3AGbW8tTPgI2WuL0hUF7be8QYb4oxFsQYC/Ly8lY0vyRJkiTVzccfw1lnQadOUF5rPclYK7ILZQBGAe/EGIcucdfDwOJdJU8EHqrl6SXAviGEdWo2L9m3ZkySJEmSmtaHH8Jpp8EWW8Att0D37tAqu05Wa70Cj9kdOB6YEkJ4vWasF3AFcG8I4RTgE+CPACGEAuCMGOOpMcbZIYR+wKs1z+sbY5zdoH8CSZIkSfo5770HAwfCnXdC69Zwxhlw8cWw0UbLf26GCTHWeklaqgoKCuLEiRPTjiFJkiQpm739NgwYAPfcA6uskhS3oiLYYIO0ky1XCGFSjLFg6fEVmYGTJEmSpOwxZQr07w/33QerrQYXXQQXXgjr1rZxfnaxwEmSJElqHiZPhn794MEHYc01oVcvOO886NAh7WQNxgInSZIkKbu98kpS3P7xD2jXDi6/HM49F9ZZJ+1kDc4CJ0mSJCk7vfgi9O0LJSXQvn2ybPLss2HttdNO1mgscJIkSZKyyzPPJMXtqacgLw+uvBLOPDNZNtnMWeAkSZIkZb4Y4cknk6WSzz4L668PQ4bA6afD6qunna7JWOAkSZIkZa4Y4dFHk+L20kvQqRNcey2ceirk5qadrsll17HjkiRJklqGGOHvf4eddoIDDoCyMhgxAj74AP7ylxZZ3sACJ0mSJCmTLFoE48fDDjvAIYfA7Nlwyy0wbVpyEPcqq6SdMFUWOEmSJEnpq66GcePgN7+BI46AefPg9tuhtBROOQXatk07YUawwEmSJElKz8KFcOedsO220K1bUuTGjoV33oETToDWbtuxJAucJEmSpKZXVQWjR8PWW8Pxx0ObNnDvvTB1KnTvDjk5aSfMSNZZSZIkSQ1qwuQyiktKKa+opGO7XIoK8+napVNy5/ffJ0sjBw6E6dOhSxd48MHkerdWzi8tjwVOkiRJUoOZMLmMnuOnUFlVDUBZRSU9x0+h1YLvOOS1ErjiCvj002R3yeuuS3aYDCHl1NnDAidJkiSpwRSXlP5Q3gBWrfqObhNL2PXa8fDNV7DbbnDzzbDvvha3lWCBkyRJktRgyisqAVjt+0qOnfwverw6nrx5Fby08XbkPXgv7LWXxa0eLHCSJEmSGswWuZE/PH0/p77yIL+o/IbnNunMnw/tRtl2O/LC3nunHS/rWeAkSZIk1V9FBQwfziNDh9D2mzk8vflvGb5bN17rtDW5bXIYVJifdsJmwQInSZIkaeXNng3XXAPDhsE339D2kEP49x970PuzXMorKum09C6UqhcLnCRJkqS6mzULhg5NdpL89ls44gjo3Rs6d+b3wAtp52umLHCSJEmSVtwXX8DgwTBiBFRWwlFHJcVt223TTtYiWOAkSZIkLV9ZGRQXw8iRyWHc3btDr16w9dZpJ2tRLHCSJEmSlu2TT+DKK+GWW6C6Gk44AXr2hC23TDtZi2SBkyRJkvRTH30EgwbBbbclt08+GS69FDbbLNVYLZ0FTpIkSdJ/TZsGAwfCHXdATg706AEXXwwbb5x2MmGBkyRJkgTwzjswYADcfTe0bQtnnw1FRdDJ7f8ziQVOkiRJasmmToX+/eHeeyE3Fy64AC68ENZfP+1kqoUFTpIkSWqJXn8d+vWD8eNhjTWS69vOPx/y8tJOpp9hgZMkSZJakldfTYrb3/8Oa68Nf/0rnHsutG+fdjKtAAucJEmS1BK89BL07QuPPgrrrJOUuLPPhnbt0k6mOrDASZIkSc3Zs88mZe2JJ6BDh+RogLPOgrXWSjuZVoIFTpIkSWpuYoSnn05m3J55BtZbDwYPhjPOgNVXTzud6sECJ0mSJDUXMcJjjyXF7cUXoWNHGDYMTjst2WFSWc8CJ0mSJC1lwuQyiktKKa+opGO7XIoK8+naJYPPQ4sRHnkkKW6vvgobbQQ33AAnnwyrrpp2OjUgC5wkSZK0hAmTy+g5fgqVVdUAlFVU0nP8FIDMK3GLFsFDDyXXuE2eDJttBjfdBCeemBzGrWanVdoBJEmSpExSXFL6Q3lbrLKqmuKS0pQS1aK6Ojl4u3NnOPxwmDsXRo+G0tJkuaTlrdmywEmSJElLKK+orNN4k6quhrvugu22g6OPhqoquPNOeOcdOOkkaNMm7YRqZBY4SZIkaQkd29W+2ceyxptEVRXcfjtsvTUceyzk5MC4cTB1anK7tVdGtRQWOEmSJGkJRYX55LbJ+dFYbpscigrzmz7M99/DLbdAfn4yw7b66vDAA/DGG3DUUUmRU4tiVZckSZKWsHijklR3oVywAG69Fa64Aj75BHbcMTkO4KCDIISmy6GMY4GTJEmSltK1S6d0dpysrISbb4arroKyMth1Vxg5EgoLLW4CLHCSJElS+ubNgxtvhOJimDED9tgjueZt770tbvqR5Ra4EMKtwEHAzBjjtjVj44DFi4DbARUxxs61PHc6MBeoBhbGGAsaKLckSZKU/ebOTQ7cHjwYvvwS9tkn2Zxkzz3TTqYMtSIzcLcB1wFjFg/EGI9e/HUIYQgw52eev1eM8cuVDShJkiQ1O3PmwPDhcPXVMHs27Lcf9OkDu+2WdjJluOUWuBjjsyGETWu7L4QQgKOAvRs2liRJktQMzZ6dbEYybFhS4g4+GHr3hp12SjuZskR9r4H7HTAjxjhtGfdH4LEQQgRGxhhvquf7SZIkSdnnyy9h6FC47rpk2eRhhyXFbYcd0k6mLFPfAncMcPfP3L97jLE8hLAu8HgI4d0Y47O1PTCE0APoAbDxxhvXM5YkSZKUAWbMSK5vGzEC5s+HP/4xKW7bbZd2MmWplT7IO4TQGjgcGLesx8QYy2s+zwQeBJY5NxxjvCnGWBBjLMjLy1vZWJIkSVL6ysvh/PNhs82SmbeuXeGtt5INSixvqoeVLnDAH4B3Y4yf1XZnCGH1EMKai78G9gWm1uP9JEmSpMz26adw9tmw+ebJJiVHHw3vvgt33glbb512OjUDyy1wIYS7gZeA/BDCZyGEU2ru6sZSyydDCB1DCP+subke8HwI4Q3gFeCRGOOjDRddkiRJyhDTp8Ppp8Mvf5kcvH388fDeezB6NGy5Zdrp1IysyC6Uxyxj/KRaxsqBA2q+/hD4TT3zSZIkSZnr/fdh4EC44w5o1QpOPRUuuQQ22STtZGqm6ruJiSRJktTylJbCgAEwdiy0bQtnnQUXXwydOqWdTM2cBU6SJElaUW+9Bf37J5uR5OYmG5VcdBGsv37aydRCWOAkSZLU5CZMLqO4pJTyiko6tsulqDCfrl0yePbq9deT4vbAA7DGGsls2wUXwLrrphIn675/ajAWOEmSJDWpCZPL6Dl+CpVV1QCUVVTSc/wUgMwrIRMnQr9+8PDDsNZayRlu550Hv/hFapGy6vunBlefYwQkSZKkOisuKf2hfCxWWVVNcUlpSolq8fLLcOCBsOOO8Oyz8Le/wccfJ2UuxfIGWfL9U6NxBk6SJElNqryisk7jTeq555KS9vjjSVEbMCA5122ttdJO9oOM/v6p0TkDJ0mSpCbVsV1uncYbXYzw9NOw116wxx7wxhtQXJyc7darV0aVN8jA75+alAVOkiRJTaqoMJ/cNjk/Gsttk0NRYX7TBokRHnsMfvc72Hvv5GiAq6+Gjz5KdpZcY42mzbOCMub7p1S4hFKSJElNavFGG6ntohgj/POf0LcvvPIKbLghXHcdnHIKrLpq02Soh9S/f0pViDGmneEnCgoK4sSJE9OOIUmSpOZk0aJkN8l+/eC112CTTZIlkieeCKusknY66UdCCJNijAVLj7uEUpIkSc3bokVw333QpQscdhjMmQO33grTpkGPHpY3ZRULnCRJkpqn6mq4+27Ybjs46ihYsADuuAPefRdOPhnatEk7oVRnFjhJkiQ1LwsXwpgxsM020L07hJAUubfeguOOg9ZuA6HsZYGTJElS8/D99zBqFOTnJ9e15ebC/ffDm29Ct26Qk7P815AynL9+kCRJUnZbsABuuw0GDYKPP4bf/hYeeggOPjiZfZOaEWfgJEmSlJ2++y7Z/n+LLeCMM2CDDZLjAV59FQ45xPKmZskZOEmSJGWX+fNh5Ei46ir44ovkIO7Ro2GffSxtavYscJIkScoO334LN9wAQ4bAzJmw995wzz2w555pJ5OajAVOkiRJme2bb5KlkkOHwldfwb77Qp8+8D//k3YyqclZ4CRJkpSZvv4arr0WrrkGKirgwAOT4rbzzmknk1JjgZMkSVJm+eoruPpqGD48mX3r2hV69052l5RaOAucJEmSMsPMmcn1bTfcAPPmwZFHJsVt++3TTiZlDAucJElSMzRhchnFJaWUV1TSsV0uRYX5dO3SKe1Ytfv8cxg8GEaMSM5069YNLrsMttkm7WRSxrHASZIkNTMTJpfRc/wUKquqASirqKTn+CkAmVXiPvssOQrgpptg4UI49ljo1Qvy89NOJmUsD/KWJElqZopLSn8ob4tVVlVTXFKaUqKlfPwxnHkm/PKXyazbccdBaSncfrvlTVoOZ+AkSZKamfKKyjqNN5kPPoBBg5Ki1qoV/OlPcMklsOmm6eaSsogFTpIkqZnp2C6XslrKWsd2uSmkAd57DwYMgLFjoXXrZPbt4othww3TySNlMZdQSpIkNTNFhfnktsn50VhumxyKCpt4eeLbb0P37rD11nDffXDuufDRR8nZbpY3aaU4AydJktTMLN6oJLVdKN98E/r3h/vvh9VWg4suggsvhHXXbZr3l5oxC5wkSVIz1LVLp6bfcfK116BfP5gwAdZaK9lR8rzzoEOHps0hNWMWOEmSJNXPf/6TFLdHHoF27eD//g/OOQfWWSftZFKzY4GTJEnSynnhBejbFx57DNq3TzYq+fOfYe21004mNVsWOEmSJK24GOGZZ5Li9vTTkJeXHMZ95pmwxhppp5OaPQucJEmSli9GeOKJZKnkc8/B+uvD0KFw+unJRiWSmoQFTpIkScsWIzz6aDLj9vLL0KkTDB8Op5wCuSmdKye1YJ4DJ0mSpJ+KER5+GHbaCQ44AD7/HG68ET74AM4+2/ImpcQCJ0mSpP9atAgeeAC6dIFDD4XZs2HUKJg2LVkuucoqaSeUWjQLnCRJkqC6Gu65B7bfHo48EiorYcwYKC2FP/0J2rRJO6EkLHCSJEkt28KFcMcd8OtfwzHHJEsn77oL3n4bjj8eWrtlgpRJLHCSJEktUVUV3HorbLUVnHBCsjTyvvtgypSkyOXkpJ1QUi38lYokSVJLsmAB3H47DBoE06fDDjvAhAlw8MHQyt/tS5nO/0olSZJagu++g+uvhy22SDYjWW89+Mc/YOLEZLMSy5uUFZb7X2oI4dYQwswQwtQlxv4vhFAWQni95uOAZTx3vxBCaQjh/RDCpQ0ZXJIkSStg/ny45hrYfPNk+/9NNoGSEnjpJTjwQAgh7YSS6mBFftVyG7BfLeNXxxg713z8c+k7Qwg5wPXA/sA2wDEhhG3qE1aSJEkr6NtvobgYNtsMzj8f8vPhqafguedg330tblKWWm6BizE+C8xeidfeCXg/xvhhjPF74B7g0JV4HUmSJK2ob75Jrm/bdFO4+OLkWIBnn4Wnn4a99rK4SVmuPoudzw4hvFmzxHKdWu7vBHy6xO3PasZqFULoEUKYGEKYOGvWrHrEkiRJaoEqKqBv36S49eoFO++cLJN8/HH43e/STiepgaxsgRsB/BLoDHwODKnlMbX9eicu6wVjjDfFGAtijAV5eXkrGUuSJKmF+eor6NMnubbt8sthjz3g1VfhkUdgl13STiepga3UMQIxxhmLvw4h3Az8o5aHfQZstMTtDYHylXk/SZIkLWXWLBgyJNlZ8ttv4YgjoHdv6Nw57WSSGtFKFbgQwgYxxs9rbh4GTK3lYa8CW4YQNgPKgG5A95VKKUmSpMQXX8DgwTBiBFRWwtFHw2WXwbbbpp1MUhNYboELIdwN/B7oEEL4DLgc+H0IoTPJksjpwOk1j+0I3BJjPCDGuDCEcDZQAuQAt8YY32qUP4UkSVJzV1YGV10FN90E338Pxx6bXOu21VZpJ5PUhEKMy7wsLTUFBQVx4sSJaceQJElK3yefwBVXwKhRsGgRnHAC9OyZHMgtqdkKIUyKMRYsPb5SSyglSZLUyD78MDkO4Pbbk9snnwyXXpqc6yapxbLASZIkZZL33oOBA+HOO6F1a+jRAy65BDbaaPnPldTsWeAkSZIywTvvwIABcPfd0LYt/OUvUFQEHTumnUxSBrHASZIkpWnKFOjfH+67D3Jz4YIL4MILYf31004mKQNZ4CRJktIweTL06wcPPghrrplc33b++ZCXl3YySRnMAidJktSUXn01KW5//zusvTb89a9w7rnQvn3aySRlAQucJElSU3jxxaS4PfoorLNO8vXZZ0O7dmknk5RFLHCSJEmN6ZlnkrL25JPQoUNypttZZyXLJiWpjixwkiRJDS1GeOop6NsXnn0W1lsPBg+GM86A1VdPO52kLGaBkyRJaigxQklJMuP24ovJEQDDhsFppyU7TEpSPbVKO4AkSVLWizHZlGTnnWH//eHTT+GGG+CDD+CccyxvkhqMBU6SJGllLVqUHAPw29/CIYfAl1/CzTfD++/DmWfCqqumnVBSM2OBkyRJqqvqarj3XvjNb+Dww2HuXBg9GkpL4dRToW3btBNKaqYscJIkSStq4UIYOxa23RaOPjq5fccd8M47cNJJ0KZN2gklNXMWOEmSpOWpqoLbboNttoHjjoPWrWHcOJg69b+3JakJ+LeNJEnSsnz/PYwZAwMHwkcfQefOMH48HHootPL34JKann/zSJIkLW3BAhgxArbcMjkCoEMHePhheO01OOwwy5uk1DgDJ0mStFhlZbKL5JVXQnk57LorjBwJhYUQQtrpJMkCJ0mSxLx5cOONUFwMM2bAnnsmSyf33tviJimjWOAkSVLLNXducuD24MHJGW777JNsTrLnnmknk6RaWeAkSVLLM2cODB8OV18Ns2fDfvtBnz6w225pJ5Okn2WBkyRJLcfs2TBsWPIxZw4cfDD07g077ZR2MklaIRY4SZLU/H35JQwdCtddlyybPPzwpLh16ZJ2MkmqEwucJElqvmbMSK5vGzEC5s+Ho46Cyy6D7bZLO5kkrRQLnCRJan7Ky5MdJUeOTM50O+aYpLhtvXXaySSpXixwkiSp+fj00+QMt1tugYUL4fjjoVev5EBuSWoGLHCSJCn7TZ8OgwbB6NHJ7ZNOgksvhc03TzOVJDU4C5wkScpe77+fFLcxY6BVKzjtNLjkEth447STSVKjsMBJkqTsU1oKAwbA2LHQti38+c9QVASdOqWdTJIalQVOkiRlj7fegv79Ydw4yM2F88+Hiy6C9ddPO5kkNQkLnCRJynxvvAH9+sEDD8AaayTLJC+4APLy0k4mSU3KAidJkjLXpElJcXvoIVhrLejTB849F37xi7STSVIqLHCSJCnzvPwyXxRdxvrPP0XFqmtw3x9OomOfIg7cY5u0k/1gwuQyiktKKa+opGO7XIoK8+naxWvwJDUuC5wkScoczz2XzLg9/jir5K7FlXueyB1dDuTbVVYj9/FPqFpz7YwoSRMml9Fz/BQqq6oBKKuopOf4KQAZkU9S89Uq7QCSJKmFixGefhr22gv22APeeIPr9j+d3c8YxYhd/si3q6wGQGVVNcUlpSmHTRSXlP5Q3hbLpHySmi8LnCRJSkeM8NhjSWnbe+/kaIBrroGPPmLI9gczv23uT55SXlGZQtCfWlaOTMknqfmywEmSpKYVIzzyCOy6KxQWwscfw/XXw4cfJhuUrLYaHdv9tLwByxxvapmeT1LzZYGTJElNY9EimDABCgrgoINgxgy46SZ4/3046yxYddUfHlpUmE9um5wfPT23TQ5FhflNnbpWmZ5PUvPlJiaSJKlxLVqUnN/Wvz+8+SZssQXceiscdxy0aVPrUxZvBJKpuzxmej5JzVeIMaad4ScKCgrixIkT044hSZLqo7oa7r03KW5vvw1bbQWXXQbdukFrf4csST8nhDApxliw9LhLKCVJUsNauBDGjIFttoHu3SEEuOcemDo1mXWzvEnSSltugQsh3BpCmBlCmLrEWHEI4d0QwpshhAdDCO2W8dzpIYQpIYTXQwhOqUmS1Jx9/z2MGgX5+XDiiZCbC/ffnyybPPpoyMlZ/mtIkn7WiszA3Qbst9TY48C2McbtgfeAnj/z/L1ijJ1rm/6TJEnNwIIFcOON8KtfwamnQvv28NBDMHkyHHEEtHLBjyQ1lOX+jRpjfBaYvdTYYzHGhTU3XwY2bIRskiQpk1VWwvDh8MtfwplnwgYbwD//Ca+8AocckiydlCQ1qIb4ldifgH8t474IPBZCmBRC6NEA7yVJktI2bx4MHQqbbw7nnJN8fvxxePFF2H9/i5skNaJ6XUUcQrgMWAiMXcZDdo8xlocQ1gUeDyG8WzOjV9tr9QB6AGy88cb1iSVJkhrD3LkwYgQMHgyzZsHeeyebk+y5Z9rJJKnFWOkZuBDCicBBwLFxGWcRxBjLaz7PBB4EdlrW68UYb4oxFsQYC/Ly8lY2liRJamhz5sCAAbDppnDJJbDDDvD88/Dkk5Y3SWpiKzUDF0LYD7gE2DPGOH8Zj1kdaBVjnFvz9b5A35VOKkmSmtbXX8OwYclHRQUcdBD06QM7LfP3sZKkRrbcAhdCuBv4PdAhhPAZcDnJrpOrkCyLBHg5xnhGCKEjcEuM8QBgPeDBmvtbA3fFGB9tlD+FJElqOF9+CVdfnWxQMncudO0KvXvDb3+bdjJJavGWW+BijMfUMjxqGY8tBw6o+fpD4Df1SidJkprOjBkwZAjccAPMnw9HHpkUt+23TzuZJKlGvTYxkSRJzcDnn0NxcXKW24IF0K0bXHYZbLNN2skkSUuxwEmS1FJ99hlceSXcfDMsXAjHHQe9eiUHckuSMpIFTpKkRjJhchnFJaWUV1TSsV0uRYX5dO3SKe1YMH06XHEFjB4NixbBiSdCz57JgdySpIxmgZMkqRFMmFxGz/FTqKyqBqCsopKe46cApFfiPvgABg6EMWOgVSv405/g0kthk03SySNJqrOVPgdOkiQtW3FJ6Q/lbbHKqmqKS0qbPkxpaTLLlp8PY8fCmWcmZW7ECMubJGUZZ+AkSWoE5RWVdRpvFG+9lRzAPW4crLIKnHsuXHQRbLBB02WQJDUoZ+AkSWoEHdvl1mm8Qb3xBvzxj7DddvDww0lpmz49OSLA8iZJWc0CJ0lSIygqzCe3Tc6PxnLb5FBUmN94bzppUnLodufOUFKS7Cg5fXqy0+S66zbe+0qSmoxLKCVJagSLNyppkl0o//Mf6NcPHnkE2rWD//s/OOccWGedhn8vSVKqLHCSJDWSrl06Ne6Ok88/nxS3xx6D9u2T693+/GdYe+3Ge09JUqoscJIkZZMY4ZlnoG9fePppyMtLlkieeSasuWba6SRJjcwCJ0lSNogRnngimXF77jlYf30YOhR69IDVV087nSSpiVjgJEnKZDHCv/6VFLeXX4ZOnWD4cDjlFMhtgh0tJUkZxV0oJUnKRDEmRwDsuCMceCCUl8ONNyYHcJ99tuVNklooC5wkSZlk0SJ44AHo0gUOPRS+/hpuuQWmTYPTT08O5JYktVgWOEmSMkF1NdxzD2y/PRx5JFRWwu23Q2lpslyybdu0E0qSMoAFTpKkNC1cCHfeCb/+NRxzTDIDd9dd8PbbcMIJ0NrL1SVJ/2WBkyQpDVVVMHo0bLUVHH98sjTy3nth6tSkyOXkpJ1QkpSB/LWeJElNafyNhdEAACAASURBVMGCZGnkoEEwfTrssAM8+CAccgi08veqkqSf5/8pJElqCt99B9dfD1tskWxGsu668I9/wMSJ0LWr5U2StEKcgZMkqTHNnw833wxXXgmffw677w6jRsH//i+EkHY6SVKWscBJktQYvv02ObetuBhmzoTf/x7Gjk0+W9wkSSvJAidJWqYJk8soLimlvKKSju1yKSrMp2uXTmnHymzffJMslRwyBL76Kplp69MHfve7tJP9hD9fSco+FjhJUq0mTC6j5/gpVFZVA1BWUUnP8VMA/Ed+bSoq4Npr4ZprksO3DzggKW677JJ2slr585Wk7OQV05KkWhWXlP7wj/vFKquqKS4pTSlRhvrqq6SobbIJXH55MtP26qvwyCMZW97An68kZStn4CRJtSqvqKzTeIsza1ayTPL665Pr3Y44Anr3hs6d0062Qvz5SlJ2cgZOklSrju1y6zTeYnzxBVx0EWy6KVx1FRx0EEyZAvffnzXlDfz5SlK2ssBJkmpVVJhPbpucH43ltsmhqDA/pUQpKyuDc8+FzTaDq69OZtzefhvuvhu23TbtdHXmz1eSspNLKCVJtVq8kUWL36Xw44+TM9xGjYJFi+D446FXr+RA7izmz1eSslOIMaad4ScKCgrixIkT044hSWrJPvwQBg2C225Lzm07+WS49NJkBk6SpEYWQpgUYyxYetwZOEmSlvTeezBwINx5J7RuDaefDpdcAhttlHYySZIscJIkAcn1bAMGwD33wCqrwF/+AkVF0LFj2skkSfqBBU6S1LJNmQL9+8N990FuLlx4YfKx3nppJ5Mk6ScscJKklmnyZOjXDx58ENZcM7m+7fzzIS8v7WSSJC2TBU6S1LK88kpS3P7xD1h7bfjrX5PjAdq3TzuZJEnLZYGTJLUML7yQFLeSkqSs9euXXOe29tppJ5MkaYVZ4CRJzdszz0DfvvDUU9ChA1xxBZx1VrJsUpKkLGOBkyQ1PzHCk08ms2zPPptsSDJkSHIkwOqrp51OkqSVZoGTJDUfMcKjjybF7aWXkiMAhg2D005LdpiUJCnLtUo7gCRJ9RYjPPww7LQTHHAAlJXBDTfABx/AOedY3iRJzYYFTpKUvRYtggcegB12gEMPha++gptvhmnT4MwzYdVV004oSVKDssBJkrJPdTWMGwfbbw9HHgnz5sFtt0FpKZx6KrRtm3ZCSZIaxQoVuBDCrSGEmSGEqUuMtQ8hPB5CmFbzeZ1lPPfEmsdMCyGc2FDBJUkt0MKFcOedsO220K1bMgM3diy8/TaceCK0aZN2QkmSGtWKzsDdBuy31NilwJMxxi2BJ2tu/0gIoT1wObAzsBNw+bKKniRJy1RVBaNHw9Zbw/HHJ0Vt3DiYMgW6d4fW7sklSWoZVqjAxRifBWYvNXwocHvN17cDXWt5aiHweIxxdozxa+BxfloEJUmq3fffJ9e0/epX8Kc/JWe3jR8Pr78ORx0FOTlpJ5QkqUnV5xq49WKMnwPUfF63lsd0Aj5d4vZnNWOSJC3bd98lu0husQX06AF5efD3v8OkSXDYYdDKS7glSS1TY685CbWMxVofGEIPoAfAxhtv3JiZJEmZav78ZMbtqqugvBx22y25ve++EGr7X4okSS1LfX6FOSOEsAFAzeeZtTzmM2CjJW5vCJTX9mIxxptijAUxxoK8vLx6xJIkZZ1582DwYNh8czjvPNhyS3jySXj+eSgstLxJklSjPgXuYWDxrpInAg/V8pgSYN8Qwjo1m5fsWzMmSRLMnQtXXAGbbgpFRbDddvDMM/Dvf8Pee1vcJElayooeI3A38BKQH0L4LIRwCnAF8L8hhGnA/9bcJoRQEEK4BSDGOBvoB7xa89G3ZkyS1JJVVEC/frDJJtCzJxQUwAsvwOOPwx57pJ1OkqSMFWKs9ZK0VBUUFMSJEyemHUOS1NBmz4ZrroFhw+Cbb+Dgg6FPH9hxx7STSZKUUUIIk2KMBUuPe3COJKnxzZoFQ4fCddfBt9/C4YdD797QpUvaySRJyioWOElS4/nii2RzkhEjoLIyObvtssuSa90kSVKdWeAkSQ2vvDw5CmDkyOQw7mOOSYrb1lunnUySpKxmgZMkNZxPPoErr4RRo2DhQjj+eOjVKzkWQJIk1ZsFTpJUfx99BIMGwW23JbdPOgkuvTQ5102SJDUYC5wkaeW9/z4MHAhjxkBODpx2GlxyCWy8cdrJJElqlixwkqS6e/ddGDAA7roL2raFs89ODuLu1CntZJIkNWsWOEnSips6Ffr3h3vvhdxcuOACuPBCWH/9tJNJktQiWOAkScv3+uvQrx+MHw9rrJEsk7zgAsjLSzuZJEktigVOkrRsEycmxe3hh2GttaBPHzj3XPjFL9JOJklSi2SBkyT91EsvJcXtX/+CddaBv/0NzjkH2rVLO5kkSS2aBU6S9F/PPQd9+8ITT0CHDsnRAGedlcy+SZKk1FngJKmlixGefjqZcfv3v2HddaG4GM44I7neTZIkZQwLnCS1VDHCY48lxe2FF2CDDeCaa5Kz3FZbLe10kiSpFq3SDiBJamIxwiOPwC67wH77wSefwPXXw4cfJhuUWN4kScpYFjhJaikWLYIJE6CgAA46CGbOhJtugvffT65zW3XVtBNKkqTlsMBJUnO3aBHcdx907gyHHQbffAOjR8N77yXLJdu2TTuhJElaQRY4SWquqqvhrrtgu+3gqKOgqgruuAPeeQdOOgnatEk7oSRJqiMLnCQ1NwsXwpgxsM02cOyxEALccw9MnQrHHQet3b9KkqRsZYGTpObi++9h1CjIz4cTT4TcXHjgAXjzTTj6aMjJSTuhJEmqJwucJGW7BQvgxhthyy3h1FOhfXt4+GGYPBkOPxxa+Ve9JEnNhetoJClbVVbCLbfAlVdCWVlyLMDIkVBYmCyblCRJzY4FTpKyzbx5SVErLoYvvoDf/Q5uuw322cfiJklSM2eBk6RsMXcujBgBgwfDrFmw997J5iR77pl2MkmS1EQscJKU6ebMgeuug6FDYfbsZIlknz6w++5pJ5MkSU3MAidJmerrr2HYsOSjogIOOigpbjvtlHYySZKUEgucJGWaL7+Eq6+G4cOTZZOHHQa9e8MOO6SdTJIkpcwCJ0mZYuZMGDIErr8e5s+HP/4RLrsMtt8+7WSSJClDWOAkKW2ff57sKHnjjcmZbt26JcVtm23STiZJkjKMBU6S0vLZZ8kZbjffDAsXwnHHQa9e8KtfpZ1MkiRlKAucJDW16dPhiitg9GhYtAhOOgkuvRR++cu0k0mSpAxngZOkpvLBBzBoENx+O7RqBaecApdcAptsknYySZKUJSxwktTYSkth4EAYOxbatIGzzoKiIthwQyZMLqP47qcor6ikY7tcigrz6dqlU9qJs8aEyWUUl5T6/ZMktRgWOElqLG+9BQMGwLhxsMoqcO65cNFFsMEGQFI+eo6fQmVVNQBlFZX0HD8FwBKyAvz+SZJaolZpB5CkZueNN5IjALbbDh5+OJltmz49OSKgprwBFJeU/lA+Fqusqqa4pLSJA2cnv3+SpJbIGThJaiiTJkG/fvDQQ7DWWslRAOedB7/4Ra0PL6+orNO4fszvnySpJXIGTpLq6z//gYMOgoICeOYZ+Nvf4OOPkzK3jPIG0LFdbp3G9WN+/yRJLZEFTpJW1vPPQ2Eh7LILvPxyslHJxx/DX/8K7dot9+lFhfnktsn50VhumxyKCvMbK3Gz4vdPktQSuYRSkuoixmSWrW9fePppWHdduOoqOPNMWGONOr3U4o023EVx5fj9kyS1RCHGmHaGnygoKIgTJ05MO4Yk/VeM8MQTSXF7/vlkM5KLL4YePWC11dJOJ0mSmpkQwqQYY8HS4y6hlKSfEyP885+w666w777JbpLXXQcffphsUGJ5kyRJTcgCJ0m1iTHZTXLHHeHAA+GLL+DGG+H99+HPf4ZVV007oSRJaoFWusCFEPJDCK8v8fFNCOG8pR7z+xDCnCUe89f6R5akRrRoEdx/P3TpAl27QkUFjBoF06bB6acnB3JLkiSlZKU3MYkxlgKdAUIIOUAZ8GAtD30uxnjQyr6PJDWJ6mq4914YMADeegvy82HMGDjmGGjtfk+SJCkzNNQSyn2AD2KMHzfQ60lS01i4EO64A379a+jePVk6effdSYk7/njLmyRJyigNVeC6AXcv475dQwhvhBD+FUL49bJeIITQI4QwMYQwcdasWQ0US5KWoaoKbr0VttoKTjghWRp5330wZQp06wY5Oct/DUmSpCZW7wIXQmgLHALcV8vdrwGbxBh/AwwHJizrdWKMN8UYC2KMBXl5efWNJUm1W7AARo6ELbeEU05JDtyeMAEmT4Yjj4RW7u0kSZIyV0P8S2V/4LUY44yl74gxfhNj/Lbm638CbUIIHRrgPSWpbr77Ltn+f4st4IwzYP314ZFH4NVX4dBDLW6SJCkrNMTFHcewjOWTIYT1gRkxxhhC2ImkMH7VAO8pSStm/vxkxq24GD7/HP7nf5Klk3/4A4SQdjpJkqQ6qVeBCyGsBvwvcPoSY2cAxBhvBI4EzgwhLAQqgW4xxlif95SkFfLttzBiBAweDDNnwu9/D3fdBXvuaXGTJElZq14FLsY4H/jFUmM3LvH1dcB19XkPSaqTb75JlkoOHQpffQX77gt9+iQzb5IkSVnO/bElNQ9ffw3XXgvXXJMcvn3ggUlx23nntJNJkiQ1GAucpOz21Vdw9dUwfHgy+3booUlx++1vAZgwuYziklLKKyrp2C6XosJ8unbplHJoSZKklWOBk5SdZs6EIUPghhtg3jw44gjo3Rt+85sfHjJhchk9x0+hsqoagLKKSnqOnwJgiZMkSVnJfbMlZZfPP4cLL4RNN012ljz44OTw7fvu+1F5AyguKf2hvC1WWVVNcUlpEwaWJElqOM7AScoOn30GV10FN98MVVXQvTv06gVbbbXMp5RXVNZpXJIkKdNZ4CRlto8/hiuuSM5uW7QITjgBevZMDuRejo7tcimrpax1bJfbGEklSZIanUsoJWWmDz+E005LitqoUXDyyTBtWvL1CpQ3gKLCfHLb5PxoLLdNDkWF+Y2RWJIkqdE5Aycps7z3HgwcCHfeCa1bwxlnwMUXw0Yb1fmlFm9U4i6UkiSpubDAScoMb78NAwbAPffAKqvAOefARRdBx471etmuXTpZ2CRJUrNhgZOUrjffhP794f77YbXVkh0mL7wQ1lsv7WSSJEkZxwInKR2TJ0O/fvDgg7DmmsnGJOefDx06pJ1MkiQpY1ngJDWtV15Jits//gFrrw2XX54sl2zfPu1kkiRJGc8CJ6lpvPgi9O0LJSVJWevfH84+OylxkiRJWiEWOEmN65lnkuL21FOQl5ec6XbWWcmySUmSJNWJBU5Sw4sRnnwyWSr57LPJhiRDhsDpp8Pqq6edTpIkKWtZ4CQ1nBiTJZJ9+8JLLyVHAFx7LZx6KuTmpp1OkiQp67VKO4CkZiBG+PvfYaedYP/9oawMbrgBPvgA/vIXy5skSVIDscBJWnmLFsH48bDDDnDIIfDVV3DzzTBtGpx5Jqy6atoJJUmSmhULnKS6q66GcePgN7+BI46AefNg9GgoLU2WS7Ztm3ZCSZKkZskCJ2nFLVwIY8fCtttCt25JkRs7Ft5+G046Cdq0STuhJElSs2aBk7R8VVVw222w9dZw3HFJURs3DqZMge7dobX7IUmSJDUF/9Uladm+/x5uvx0GDoTp06FLl+Sat0MPhVb+/keSJKmp+S8wST/13XfJLpJbbAE9esC66ya7TE6aBIcdZnmTJElKiTNwkv6rshJuugmuugrKy2G33ZJdJffdF0JIO50kSVKLZ4GTlOwieeONUFwMM2bAnnvCHXfAXntZ3CRJkjKIBU5qyebOheuvhyFD4Msv4Q9/gHvvhT32SDuZJEmSamGBk1qiigoYPhyuuQZmz4b994c+fWDXXdNOJkmSpJ9hgZNaktmzYdiw5GPOHDjkEOjdG3bcMe1kkiRJWgEWOKkl+PJLGDoUrrsuWTZ5+OFJcevSJe1kkiRJqgMLnNSczZgBgwcnRwJUVsJRR8Fll8F226WdTJIkSSvBAic1R+XlyVEAI0cmh3F37w69esHWW6edTJIkSfVggZOak08+gSuvhFGjYOFCOOEE6NkTttwy7WSSJElqABY4qTn46CO44goYPTq5ffLJcOmlsNlm6eaSJElSg7LASdns/fdh4EAYMwZycuC00+CSS2DjjdNOJkmSpEZggZOy0bvvwoABcNdd0LYtnH02FBVBp05pJ5MkSVIjssBJ2WTq1KS4jRsHublwwQVw4YWw/vppJ5MkSVITsMBJ2eD116F/f3jgAVhjjeT6tvPPh7y8tJNJkiSpCVngpEw2cSL06wcPPwxrrw19+sB550H79mknkyRJUgoscFImeumlpLj961+wzjrQty/85S/Qrl3aySRJkpQiC5yUSZ57LilrTzwBHTrAoEFw1lmw1lppJ5MkSVIGsMBJaYsRnn46mXH7979hvfVg8GA44wxYffW000mSJCmD1LvAhRCmA3OBamBhjLFgqfsDMAw4AJgPnBRjfK2+7ytlvRjhsceS4vbCC9CxI1xzTXKW22qrpZ1OkiRJGaihZuD2ijF+uYz79ge2rPnYGRhR81ktxITJZRSXlFJeUUnHdrkUFebTtUvmnFfW5PlihEceSYrbK6/ARhvB9dfDn/4Eq67aeO8rSZKkrNcUSygPBcbEGCPwcgihXQhhgxjj503w3krZhMll9Bw/hcqqagDKKirpOX4KQEaUuCbNt2hRsptk374weTJsthncdBOceGJyGLckSZK0HK0a4DUi8FgIYVIIoUct93cCPl3i9mc1Y2oBiktKfyhHi1VWVVNcUppSoh9rknyLFsF990HnznDYYTB3LoweDaWlyXJJy5skSZJWUEPMwO0eYywPIawLPB5CeDfG+OwS94danhOXHqgpfz0ANt544waIpUxQXlFZp/Gm1qj5qqth3LjkAO533oGttoI774Sjj4bW7h8kSZKkuqv3DFyMsbzm80zgQWCnpR7yGbDRErc3BMpreZ2bYowFMcaCvLy8+sZShujYLrdO402tUfItXAi33w7bbAPHHgs5OXDPPTB1anLb8iZJkqSVVK8CF0JYPYSw5uKvgX2BqUs97GHghJDYBZjj9W8tR1FhPrltcn40ltsmh6LC/JQS/ViD5vv+e7jlFsjPh5NOSnaSfOABeOONZNYtJ2e5LyFJkiT9nPpOBawHPJicFEBr4K4Y46MhhDMAYow3Av8kOULgfZJjBE6u53sqiyzeCCRTd6FskHwLFiTXtA0aBJ98AgUFyXEABx0EobYVxJIkSdLKCcnmkJmloKAgTpw4Me0Y0s+rrExm3K68EsrKYNdd4a9/hcJCi5skSZLqJYQwaekztqFpjhGQmpd582DkSCguhi++gD32SK5523tvi5skSZIalQVOWlFz58INN8CQITBrFuyzT7I5yZ57pp1MkiRJLYQFTlqeOXNg+HC4+mqYPTtZItmnD+y+e9rJJEmS1MJY4KRlmT0bhg1LPubMSTYl6dMHdlr6pAxJkiSpaVjgpKV9+WUy2zZ8eLJs8rDDoHdv2GGHtJNJkiSphbPASYvNmJFc33bDDTB/Pvzxj3DZZbD99mknkyRJkgALnATl5cmOkiNHJme6deuWFLdttkk7mSRJkvQjFji1XJ9+mpzhdsstsHAhHHcc9OoFv/pV2skkSZKkWlng1PJMnw5XXAG33goxwkknQc+esPnmaSeTJEmSfpYFTi3H++/DoEEwZgy0agWnngqXXAKbbJJ2MkmSJGmFWODU/JWWwoABMHYstG0LZ50FRUWw4YZpJ5MkSZLqxAKn5uutt6B/fxg3DlZdFc47Dy66CDbYIO1kkiRJ0kqxwDUDEyaXUVxSSnlFJR3b5VJUmE/XLp3SjpWeN95Iitv99/9/e/cerFVd73H8/XWDzs5LaN7CCJs8g42NCjFWYzZYKUYmnGOdIRqt9GTUobyiAmrlXVAyL1HeRs3LkRTJsTjIdBw7OWlxsTCEJMWOQGIiIbItwO/5Yz04291+uO29n/U8+3m/Zhj2Xms9e3/3/j2/tdZnr99aP9h1Vzj3XDjrLNh337Ir65V8/0mSJNWOAa7BzVywnAkzFtK2YRMAy9e0MWHGQoDmO4meNw8uuQR++lPYY49i8u0zzoB3vavsynot33+SJEm1tVPZBahrpsxe8tbJ82ZtGzYxZfaSkioqwRNPwGc+A0OHwmOPwXe/Cy+8UIQ5w1uP8v0nSZJUW16Ba3Ar1rRt1/Je5Ve/gosvhjlziqB22WUwblxx9U010dTvP0mSpBJ4Ba7B9e/Xul3LG14mPPooHH00HHVUcb/b5MnF3G4TJxreaqzp3n+SJEklM8A1uPHDB9Hat+Vty1r7tjB++KCSKuohmcWVto9/HD7xCVi8GL73PXj++WJKgN12K7vCptQ07z9JkqQ64RDKBrf5QRG99imAmTBrVjFU8skni7nbrr8eTj0VWr3KU7Ze//6TJEmqM5GZZdfwT4YOHZpz584tuwyVKRMeeqh4EMm8eTBwIEyYAF/+MuyyS9nVSZIkST0qIuZl5tCOyx1Cqfry5pvF/G2DB8OoUfDqq3DrrfDss/C1rxneJEmS1NQMcKoPmzbBvffCoYfC5z8PbW1w552wZAmccgr07Vt2hZIkSVLpDHAq18aN8OMfwyGHwJgxxdDJe++FRYvgpJOgj7dpSpIkSZsZ4FSODRvgttvg4IPh5JOLoZE/+QksXAijR0NLy9a/hiRJktRkvLyh2vr73+H22+HKK4u524YMgZkz4bOfhZ38e4IkSZK0JZ4xqzbeeANuuAEOOgjGjoX99oOf/QzmzoWRIw1vkiRJ0jbwCpx61vr1cNNNMHkyrFwJRx5ZDJ381KcgouzqJEmSpIZigFPPWLcOpk2Dq6+GVatg2DC46y44+miDmyRJkrSDDHDqXmvXFkMlp06FV16BY46BCy+Eo44quzJJkiSp4Rng1D1efRWuuw6uvRbWrIERI4rg9pGPlF2ZJEmS1GsY4NQ1r7xShLbrriuuvo0cWQS3D32o7MokSZKkXscApx2zalUxTPLGG+H11+HEE+GCC+Cww8quTJIkSeq1DHDbYOaC5UyZvYQVa9ro36+V8cMHMWrwAWWXVY6VK4sHk0ybVkwNMHo0TJoEhxxSdmWSJElSr2eA24qZC5YzYcZC2jZsAmD5mjYmzFgI0Fwh7sUXi6kAbr4ZNmyAMWNg4kQ4+OCyK5MkSZKahrMnb8WU2UveCm+btW3YxJTZS0qqqMZeeAG+8Q14//uLq25jxsDixXDnnYY3SZIkqca8ArcVK9a0bdfyXuO55+CKK+D224t52045Bc4/Hw48sOzKJEmSpKZlgNuK/v1aWd5JWOvfr7WEamrgj3+Eyy8vJt3u0wfGjoVzz4UBA8quTJIkSWp6DqHcivHDB9Hat+Vty1r7tjB++KCSKuohixbBF78IH/gATJ8O3/oWPP88XH+94U2SJEmqE16B24rNDyrptU+h/P3v4dJL4f774R3vgHPOgbPPhn33LbsySZIkSR0Y4LbBqMEH9J7AttmCBXDJJfDgg7D77jBhApx5Juy9d9mVSZIkSarCANdsfvObIrg9/DD06wff/jacfjrsuWfZlUmSJEnaCgNcs3j88SK4zZ4Ne+1VDJscNw7e+c6yK5MkSZK0jXb4ISYRMSAiHo2IZyLiDxFxeifbDIuIv0XEU5V/F3WtXG23xx6DT34SPvYxmD8frroKli2DSZMMb5IkSVKD6coVuI3A2Zk5PyJ2B+ZFxJzMXNRhu//NzOO78H20vTLhF78orrj98pew//4wdSqcdhrsumvZ1UmSJEnaQTt8BS4zV2bm/MrHrwHPAL3sSR8NJhNmzYIjj4RjjoE//amYBuC554oHlBjeJEmSpIbWLfPARcSBwGDgyU5WfzQifhcRsyLikC18jdMiYm5EzH355Ze7o6zmkQkPPQRHHAEjRsDy5TBtWhHgxo2D1l466bgkSZLUZLoc4CJiN+AB4IzMXNth9XxgYGYeBlwPzKz2dTLzpswcmplD99lnn66W1RzefBNmzIAhQ2DkSFi9Gm65BZ59FsaOhV12KbtCSZIkSd2oSwEuIvpShLe7M3NGx/WZuTYz11U+/jnQNyKcaKyrNm2C++6Dww6DE0+E9evhjjtgyRI49VTYeeeyK5QkSZLUA7ryFMoAbgWeycypVbbZv7IdEXFE5fu9sqPfs+lt3Ah33QUf/CCMHl0EuXvugUWL4OSToY+zQkiSJEm9WVfO+I8ETgIWRsRTlWUTgfcCZOYPgc8BX4+IjUAbMDozswvfszlt2AB33w2XXQZLl8Khh8L06cXVt5265TZGSZIkSQ1ghwNcZv4KiK1scwNww45+j6b3j38UQyMvv7yYu23IEHjwQTjhBIObJEmS1IRMAfXojTfgBz+Agw4q5m7bbz94+GGYOxdGjTK8SZIkSU3Km6bqyfr1cPPNMHkyrFhRzOd2yy3FnG6xxYudkiRJkpqAAa4evP56MW/b1VfDSy/BsGHFw0qGDTO4iZkLljNl9hJWrGmjf79Wxg8fxKjBB5RdliRJkkpggCvTa6/BjTfCNdfAX/9aXGm78EI46qiyK1OdmLlgORNmLKRtwyYAlq9pY8KMhQCGOEmSpCbkzVRlWLMGLrkEBg6ECRPgiCPg17+GRx4xvOltpsxe8lZ426xtwyamzF5SUkWSJEkqk1fgamn1arj2Wvj+92Ht2uJpkhdeCEOHll2Z6tSKNW3btVySJEm9mwGuFl5+GaZOhRtugHXrivnbLrgADj+87MpU5/r3a2V5J2Gtf7/WEqqRJElS2RxC2ZP+8hc45xw48EC46io4/nhYuBDuv9/wpm0yfvggWvu20T0/ZwAAB31JREFUvG1Za98Wxg8fVFJFkiRJKpNX4HrCihXFVAA/+lExGfeYMTBpEhx8cNmVqcFsflCJT6GUJEkSGOC615//XFxpu/VW2LQJTjoJJk4sJuSWdtCowQcY2CRJkgQY4LrH88/DFVfA7bcXn3/lK3D++fC+95ValiRJkqTexQDXFUuXwuWXw513QksLnHYanHceDBhQdmWSJEmSeiED3I5YvBguuwzuuQd23hm++U0YPx769y+7MkmSJEm9mAFuezz9NFx6KUyfDq2tcNZZcPbZsP/+ZVcmSZIkqQkY4LbFq6/CV78KDzwAu+1W3N925pmwzz5lVyZJkiSpiRjgtsUeexRTA1x0EZx+Ouy1V9kVSZIkSWpCBrht0dICjz8OEWVXIkmSJKmJ7VR2AQ3D8CZJkiSpZAY4SZIkSWoQBjhJkiRJahAGOEmSJElqEAY4SZIkSWoQBjhJkiRJahAGOEmSJElqEAY4SZIkSWoQBjhJkiRJahAGOEmSJElqEAY4SZIkSWoQBjhJkiRJahAGOEmSJElqEAY4SZIkSWoQBjhJkiRJahAGOEmSJElqEAY4SZIkSWoQBjhJkiRJahCRmWXX8E8i4mXghbLrEHsDfy27CL3F9qgvtkd9sT3qi+1RX2yP+mJ71Jd6bo+BmblPx4V1GeBUHyJibmYOLbsOFWyP+mJ71Bfbo77YHvXF9qgvtkd9acT2cAilJEmSJDUIA5wkSZIkNQgDnLbkprIL0NvYHvXF9qgvtkd9sT3qi+1RX2yP+tJw7eE9cJIkSZLUILwCJ0mSJEkNwgDX5CJiQEQ8GhHPRMQfIuL0TrYZFhF/i4inKv8uKqPWZhERyyJiYeV3PbeT9RER10XE0oj4fUQMKaPOZhARg9q975+KiLURcUaHbewfPSgibouIVRHxdLtle0XEnIh4tvL/nlVe+6XKNs9GxJdqV3XvVaU9pkTE4sr+6MGI6FfltVvct2n7VWmP70TE8nb7pBFVXntcRCypHEvOr13VvVeV9rivXVssi4inqrzW/tHNqp3j9oZjiEMom1xEvBt4d2bOj4jdgXnAqMxc1G6bYcA5mXl8SWU2lYhYBgzNzE7nJKkcjL8JjAA+DHw/Mz9cuwqbU0S0AMuBD2fmC+2WD8P+0WMi4uPAOuDOzPxgZdlkYHVmXlk58dwzM8/r8Lq9gLnAUCAp9m0fysxXa/oD9DJV2uNY4H8yc2NEXAXQsT0q2y1jC/s2bb8q7fEdYF1mXr2F17UAfwSOAV4Efgt8of2xX9uvs/bosP4a4G+ZeXEn65Zh/+hW1c5xgS/T4McQr8A1ucxcmZnzKx+/BjwDHFBuVdqKkRQHh8zMJ4B+lZ2UetYngT+1D2/qeZn5S2B1h8UjgTsqH99BcUDuaDgwJzNXVw64c4DjeqzQJtFZe2TmI5m5sfLpE8B7al5Yk6rSP7bFEcDSzHwuM/8B/BdFv1IXbKk9IiKAfwfurWlRTWwL57gNfwwxwOktEXEgMBh4spPVH42I30XErIg4pKaFNZ8EHomIeRFxWifrDwD+r93nL2LoroXRVD/w2j9qa7/MXAnFARrYt5Nt7CflOAWYVWXd1vZt6j7jKkNab6syPMz+UXtHAS9l5rNV1ts/elCHc9yGP4YY4ARAROwGPACckZlrO6yeDwzMzMOA64GZta6vyRyZmUOATwP/WRmS0V508hrHQvegiNgZOAH4SSer7R/1yX5SYxExCdgI3F1lk63t29Q9pgHvBw4HVgLXdLKN/aP2vsCWr77ZP3rIVs5xq76sk2V100cMcCIi+lK8se/OzBkd12fm2sxcV/n450DfiNi7xmU2jcxcUfl/FfAgxVCX9l4EBrT7/D3AitpU17Q+DczPzJc6rrB/lOKlzcOGK/+v6mQb+0kNVW7wPx74Yla5uX4b9m3qBpn5UmZuysw3gZvp/Pds/6ihiOgD/BtwX7Vt7B89o8o5bsMfQwxwTa4yJvtW4JnMnFplm/0r2xERR1C8b16pXZXNIyJ2rdxoS0TsChwLPN1hs4eAk6PwEYobolfWuNRmU/Uvp/aPUjwEbH4i2JeAn3ayzWzg2IjYszKE7NjKMnWziDgOOA84ITPXV9lmW/Zt6gYd7on+Vzr/Pf8W+JeIeF9lhMFoin6lnvEpYHFmvtjZSvtHz9jCOW7DH0P6lF2ASnckcBKwsN2jbScC7wXIzB8CnwO+HhEbgTZgdLW/sKrL9gMerOSBPsA9mfnfETEW3mqPn1M8gXIpsB74Skm1NoWIeAfFk9q+1m5Z+/awf/SgiLgXGAbsHREvAt8GrgSmR8SpwJ+Bz1e2HQqMzcz/yMzVEXEJxYkqwMWZuSMPe1A7VdpjArALMKey73oiM8dGRH/glswcQZV9Wwk/Qq9SpT2GRcThFMO9llHZd7Vvj8oTQ8dRnJC2ALdl5h9K+BF6lc7aIzNvpZN7qO0fNVHtHLfhjyFOIyBJkiRJDcIhlJIkSZLUIAxwkiRJktQgDHCSJEmS1CAMcJIkSZLUIAxwkiRJktQgDHCSJEmS1CAMcJIkSZLUIAxwkiRJktQg/h9/xXPQdTpT2gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "m = 20 #设置数据集大小\n",
    "X0 = np.ones((m,1))   #m行1列\n",
    "X1 = np.arange(1,m+1).reshape(m,1) #m行1列\n",
    "X = np.hstack((X0,X1)) #沿着水平方向将矩阵堆叠起来\n",
    "\n",
    "Y= np.array([\n",
    "    3, 4, 5, 5, 2, 4, 7, 8, 11, 8, 12,\n",
    "    11, 13, 13, 16, 17, 18, 17, 19, 21\n",
    "]).reshape(m, 1)\n",
    "\n",
    "#画出原始数据散点图\n",
    "fig,ax = plt.subplots(figsize=(15,8)) \n",
    "ax.scatter(X1,Y)\n",
    "\n",
    "#用最小二乘法得出参数\n",
    "theta =  np.dot(np.dot(np.linalg.inv(np.dot(X.T, X)), X.T), Y)\n",
    "model_y = theta[0]*X0 + theta[1]*X1\n",
    "\n",
    "#画出模型->直线\n",
    "ax.plot(X1,model_y,color = 'red')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
